Installing Solr on Ubuntu Jaunty 9.04

Solr install success on Ubuntu Jaunty 9.04

I just installed Apache Solr on Ubuntu Jaunty 9.04. I am not a Java or Tomcat person, so it was all rather new and a confusing, there was no good install guide and I hit a few walls, so here are some pointers that should get you to a working install.

I am using the command-line install method, so it’s works for desktop and server editions. You can skip the apt-gets and use Synaptics Package Manager to install the packages if you prefer.

1) Install Java

sudo apt-get install sun-java6-bin sun-java6-demo sun-java6-jdk sun-java6-jre

I believe the default install of Jaunty installs openjdk java, so you will want to make sure that Sun Java is chosen:

sudo update-alternatives --config java

You will prompted to pick a version as the default:

Select the Sun Java version

Edit /etc/profile to sent the JAVA_HOME environment variable:

JAVA_HOME=/usr/lib/jvm/java-6-sun
export JAVA_HOME

2) Install Tomcat

sudo apt-get install tomcat5.5 libtomcat5.5-java tomcat5.5-admin tomcat5.5-webapps

Check that Tomcat is running:

sudo service tomcat5.5 status

Grant yourself Tomcat Manager and Admin permissions; edit /usr/share/tomcat5.5/conf/tomcat-users.xml and include the following:

<user username="myname" password="mypassword" roles="admin,manager"/>

3 Install Solr

sudo apt-get install solr-common solr-tomcat5.5 libxpp3-java

(I kept getting “java.lang.NoClassDefFoundError: org/xmlpull/v1/XmlPullParserException at java.lang.Class.forName0(Native Method)” in the Solr admin page; a little searching led me to find that the problems was that libxpp3-java was not installed. Why it’s not a required package, I don’t know!)

4) Restart Tomcat and Test Solr

sudo service tomcat5.5 restart

Now you can go to the Tomcat web admin page at http://localhost:8180/ and login with the credentials you supplied above.

Go to http://localhost:8180/solr/ and do a test query to confirm it’s working.

Of course you will get zero results, you have no content indexed yet.

That should be enough to get started!

Tidy Up Your XML in Vim

You have some nice, compact XML, the kind without whitespace or newlines, like this:

Not easy to read. OK for computers to crunch–but if you want to get an idea of the document’s structure at a glance or do some editing–well, you might want it formatted a little nicer.

So let’s use Tidy to clean it up–all from the comfort of your Vim session…

(You could also use xmllint–your choice–depends on what you have installed and what you prefer.)

You can execute a shell command from Vim by typing:

:!

Then to tidy up your XML run tidy with the following options:

tidy -mi -xml -utf8

“-im” means indent and modify the original file; “-xml” means, you are dealing with XML and “-utf8″ is optional, it means “use UTF-8 for both input and output” so you’ll get readable characters instead of encodings if you have umlats or grave symbols.

You can see all of tidy’s formatting, encoding and other options by typing

tidy -h

So the whole command we type into Vim is:

:! tidy -mi -xml -utf8

You will drop to shell while the command executes:

Hit “return” when you are done and then Vim will ask if you want to reload the file:

And there you go, beautifully formatted HTML to read!

Enjoy !

Google Chrome Changes the Game: The Browser as Platform

Google announced their new browser named “Chrome” today. Check out the blog post and, better yet, the Google Chrome comic book.

The platform wars just moved from the OS to the browser and Google took a commanding lead.

Rather than re-write everything, allow me the luxury of re-purposing my Tweets:

• Chrome is all open source. They have made the world their R&D department. Brilliant.
• Chrome just increased the importance of Javascript dramatically. They launched a platform where Javascript is the dominant language.
• Google just moved the development platform to the browser. The OS just took a backseat
• If MS was afraid that Google had them in the Search Engine Market, they should be shitting themselves about now about the browser market.
• Chrome will set the bar for what people will expect in a web browser.
• Google didn’t have to reinvent the OS; they just had to build the best browser that could run on any OS.
• Automated testing against google’s vast index of web pages is a stroke of brilliance for stability testing a browser.
• Chrome should solve the biggest annoyance I have with my browser: better memory management so I don’t have to restart my broswer 3X daily.
• Chrome is privacy-oriented. That’s a good thing.
• The name is very toungue-in-cheek: chome is a refernece to the UI for an application, Google wants Chrome to be the UI for the user’s web experience.
• This is the biggest architectural leap in computing in a long time.

Google has just moved to the forefront of the Browser wars and will force the competition to keep up. They have the brand recognition, industry leverage and exposure to get their browser installed in a LOT of computers.

Oh–an it’s also built on the same engine that the Android mobile browser will run on.

Mark my words: Google just changed the game on the web. Chrome is the lever they use to move the world.

How to Mount a VMWare (Linux) Partition in Ubuntu (Hardy 8.04)

I recently had a problem starting a VMWare VM (the VM ran CentOS) and I needed to get some data off the virtual machine. Since the VM would no longer boot, I decided to try to mount the partition under Linux and copy the data off. It was a little trickier than I had anticpated so I tought I would share my solution.

vmware-mount.pl Doesn’t Get You All The Way There

vmware-mount.pl does just what is says it will do: it mounts VMWare partitions on your computer, IF the target computer can mount the partition type. The problem with Ubuntu (Hardy 8.04) was that it would not mount an LVM partition.

Look at vmware-mount.pl for moreon how to use this command.

But vmware-mount.pl was still useful in helping me find out the partition type and creating a block device from the .vmdk file that I could use other tools to mount.

Change to the directory where you have the virtual machine files. There will be one (often more) .vmdk files in that directory and these files are the virtual drives that hold the data for your VM.

cd /path/to/your/vm/

Now let’s see what is inside these files; we want to know what partitions there are so we can figure out which one has the data we are searching for.

sudo vmware-mount.pl -p name-of-the-vmware-virtual-drive.vmdk

That will output something like:


--------------------------------------------
VMware for Linux - Virtual Hard Disk Mounter
Version: 1.0 build-80004
Copyright 1998 VMware, Inc. All rights reserved. -- VMware Confidential
--------------------------------------------

Nr Start Size Type Id Sytem
-- ---------- ---------- ---- -- ------------------------
1 63 208782 BIOS 83 Linux
2 208845 16563015 BIOS 8E Unknown


So we have two partitions in this vmware virtual drive. Just by looking at it I can tell that the big partition, partition number 2 is the one that has all my data. But notice the “Id” is “8E” and the “Sytem” is “Unknown.” Well “8E” is the ID for LVM file systems and that’s a problem; let’s try and mount it and see what happens:

Create a mount point:

sudo mkdir /mnt/vmdk

Now let’s try to mount it.

The command syntax for mounting a .vmdk partition is

sudo vmware-mount.pl name-of-the-vmware-virtual-drive.vmdk [partition number] /mnt/point

So if we want to mount partition 2 on /mnt/vmdk we’ll try:

sudo vmware-mount.pl name-of-the-vmware-virtual-drive.vmdk 2 /mnt/vmdk

And the results we get are something like this:


jb@jb-ubuntu:~/Desktop/fuxored-disk$ sudo vmware-mount.pl CentOS\ 4.4-000007-cl1.vmdk 2 /mnt/cent
[sudo] password for jb:

Unable to retrieve the filesystem of the partition (the partition type is BIOS
and the partition Id is 8E). Please file an incident with VMware at
http://www.vmware.com/forms/Incident_Login.cfm by copying this error message.
jb@jb-ubuntu:~/Desktop/fuxored-disk$


It created a block device that can be read, but the partition didn’t mount as we can see by this message:


Client: The partition is now mapped on the /dev/nb0 Network Block Device.
mount: unknown filesystem type 'LVM2_member'


So we have a block device of type “LVM2_member.” Let’s try to find a way to mount that!

Luckily, Kurt Wall did a nice tutorial to how to mount a fedora logical volume in Ubuntu that solves our problem.

If you type in the commands he has listed on that page, one after the other, you will have a mounted partition and problem solved!

Fixing Compiz Fusion on ATI Radeon Mobility 9800 Under Ubuntu Hardy (8.04)

**** OK, this works–but it’s not feasible. I found xserver-xgl to be VERY slow and had screen rendering issues. ****

Take this for what it’s worth–and that does not seem to be much.

When I originally installed Ubuntu Hardy on my HP Pavillion zt3000 laptop Compiz worked perfectly. Much to my dismay, after I re-installed, Compiz Fusion would not re-enable, even though I had the restricted drivers installed and enabled. (Selecting System -> Preferences -> Appearance -> Visual Effects -> Extra returned a “Desktop effects could not be enabled” error.)

I tried running “compiz” from the command line and one line of the output caught my eye:

checking for xgl not present

That gave me enough search terms to search on Google and hit paydirt on UbuntuForums.

The solution was simple: Go to System -> Administration -> Synaptics Package Manager and install “xserver-xgl.”

After I logged back out and back in, I enabled Compiz ( System -> Preferences -> Appearance -> Visual Effects -> Extra) and everything worked.

Tuning LAMP Systems

I have been on a website performance and scalability kick lately. I thought I would share some articles I have enjoyed on the topic.

IBM Developer Works has a nice little two part series on Tuning Lamp Systems (pt 2).

Then again, IBM Developer Works often has some awesome articles. Scan it regularly if you can!

Enjoy!

Thai Open Source Initiative Uses… .NET?

From the The Nation:

Soon, a factory will lead the push for development of open-source code software for Thai industries, locally.

The Association of Thai Software Industry (ATSI), the Industrial Promotion Department, the Software Industry Promotion Agency (Sipa), Microsoft (Thailand), Rangsit University and 20 local software companies have joined hands to set up the country’s first software factory.

ATSI president Somkiat Ungaree said the software factory is expected to open in the next two months. It will be housed at Rangsit University. The factory will receive Bt3 million in funding from the Industrial Promotion Department and Bt2 million from Sipa.

The factory’s first project will be developing a prototype of small-size manufacturing resource planning (MRP) software used in small and medium manufacturing plants.

Excellent! Home-grown, open source software so small Thai manufacturing plants won’t have to shell out big-bucks and be locked into proprietary software. GREAT IDEA!

But wait…

He said 100 programmers from the 20 local software companies, would be trained in Microsoft’s .Net platform at the factory. They will then develop open-source code software two days a week at the factory.

Huh? They are developing it in .NET! OK–so application will be open source and the application will be free… you are just locked into an expensive, closed-source, insecure PLATFORM.

I don’t know who dreams this stuff up…

Total Freedom! Laptop with Ubuntu Gutsy Gibbon Linux, Nokia N70 Modem and AIS (One-2-Call) EDGE Wireless

As I post this, I am on my laptop running Ubuntu 7.10 (Gutsy Gibbon) and connecting the Internet via AIS EDGE wireless internet with my Nokia N70 mobile phone.

One more critical step closer to completely abandoning Windows forever.

The setup was fast and relatively simple once I found Howto connect to the Internet with a Nokia N70 from Ndlovu on UbuntuForums.

For people that have my exact setup (Ubuntu Gutsy, Nokia N70 (connected via the USB cable packaged with the phone), AIS (One-2-Call) in Thailand) I can simplify the process a bit for you.

All I had to do was create the two config files as per the instructions and save them. Then go to the command line and type “pon mobile” and hit enter. Fast and simple–and it worked the first time.

Be warned–there is no feedback; you enter “pon mobile” and you get no “you are connected to the Internet” message–you just are.

If you want to see what happened during the connection process (and to see if you are really connected) try typing “plog” from the command line. Plog is part of the PPPD package (along with “pon” and “poff”) and it shows you the last few lines of /var/log/ppp.log. If that file doesn’t exist, it shows you the last few lines of your /var/log/syslog file, but excluding the lines not generated by pppd. So you can see the status messages that have been written by PPPD about your connection in ppp.log.

A few more notes:

1. PPP is part of the default install for Ubuntu Gutsy–you probably have it installed unless you customized the install to NOT install it.
2. Your user probably belongs to the dialout group already–no need to add. (But it’s not that hard to do, really.)
3. The text he suggests to cut-and-paste all worked exactly as he has it–no need to tweak the settings

It would be nice to have a graphic interface that stepped you through the process, showed you the phone was connected and had a simple little “Connect to Internet Via Mobile” and “Disconnect” buttons–but that’s just me being spoiled by how easy most things are in Ubuntu most of the time.

Ubuntu Gutsy on My Laptop: Not Winning My Heart

I installed Ubuntu Gutsy Gibbon on my laptop past week. The Wifi setup is horrible! What a terrible interface. I can’t get it to work, I can’t seem to undo my mistakes. If you can’t make Wifi simple, you are not going to win converts.

It’s ridiculous to ask the user what kind of encryption in on the access point. When I sit down at a coffee house, how do I know what kind of encryption the WAP is using? Hell, 99% of the time if you ask the staff they just give you a blank stare. No other OS vendor I know of requires you give the encryption type… Poor design.

BIND to the Rescue

I have been too busy to do much development on my own stuff with Barcamp Bangkok taking up most of my free time lately. But I am ramping up for some serious development time after Barcamp finishes.

The fact that Microsoft hosed my hosts file has meant that I cannot access my SVN repository from my desktop development environment–the only one that is on a fast computer. So I have installed BIND to run DNS on my development network. Yes, running and maintaining a DNS server is probably overkill, but MS forced me onto this corner.

BIND to the rescue! Yay! Now I have to spend a bit of time to remember how to write zone files–not that hard as I recall. There’s probably a lot of sample zone files out there already–I should search for one and set up DNS while I am on the train.

I wonder how often people have set up DNS servers while on the train from Laos to Bangkok? Probably not many. It may be a world first.