OpenSongSearch | Pragmatic Pontifications

Introduction

OpenSongSearch is a companion utility to the OpenSong lyric projection program. The purpose of OpenSongSearch (hereafter I’ll just call it OSS!) is to provide a full text search capability to OpenSong on the Windows platform (I’d be thrilled if someone would integrate it into OpenSong in RealBasic so that it would be available directly in OpenSong and be available on the other platforms).

OSS is written in AutoIt3 and uses the BaseX XML database (OSS is packaged with version 6.3.5 of BaseX) to store a copy of the OpenSong “database” of songs that can be queried against very flexibly. AutoIt3 provides the user interface and the Windows dialog manipulation utilities that allows OSS to “interface” with certain OpenSong dialogs. This allows search results to be added automatically into set lists both in set mode and projection mode.

top

History

Necessity was the mother of this invention. At our small church in France, we have a very free worship style and France has a number of standard song books that virtually all churches use. OpenSong has a decent following in France and these standard song books have all been converted to OpenSong format.

Consequently, in our services and worship tabernacles we need to be able to pull up any song at any time from a database of thousands of songs. We have worship leaders whose repertoire is somewhat dated and others who stay with more recent songs. Songs may be begun anywhere within the song and sometimes we mix and match verses from one song with another.

There are projection programs which already have full text searches but their projection interface is not as flexible for switching between songs as OpenSong and no other program has as vast a repertoire of songs in the French language. So for us OpenSong fits well except for its glaring lack of a full text search.

I originally released version 1 of OSS in 2007. Another person made some modifications and gave me some ideas to improve it. Also we wanted add more songs but the performance was suffering on the queries and the database update was reeeeealllly slooooooowwww. So I discovered the BaseX database and was impressed by its database loading speed (without having to convert the OpenSong song files since they are XML documents already) and the query speed as well. I dumped my Sqlite database and got on the the XML band wagon.

OpenSongSearch 2.0 is a complete rewrite and enhancement over the original version (assuming you ever saw it!).

top

System Requirements

I’ve tested this on Windows XP (French and English versions) and Windows 7. I believe it should work on any Windows platform which can run OpenSong. I’ve not tested it on Wine, but I have my doubts. Let me know if anyone gets it to work under Wine. System performance will depend upon system capacities, but we have it running with thousands of songs on an “average” XP box at church and it’s satisfactory.

top

Installation

Unzip the zip file to your OpenSong documents folder. The zip file has a relative path of “OpenSongSearch” in it. So the typical installation would be:

Windows XP – C:Documents and SettingsyourusernameMy DocumentsOpenSongOpenSongSearch

Vista/7 – C:UsersyourusernameDocumentsOpenSongOpenSongSearch

Execute “OpenSongSearch.exe” (OpenSongSearch.au3 is the AutoIt3 source code and can be executed if you have an installation of AutoIt3. The executable does not need an installation of AutoIt3 or any other specific .dll files)
On the first execution, OSS will try and locate the installation of OpenSong and from OpenSong’s parameters will try and set the location of OpenSong’s song files. If it succeeds it will automatically create its database (you’ll see the progress dialog if so). If it can’t find the OpenSong files, it will give you a message and you’ll have to go into the configuration and set those paths manually.

top

Basic Usage

Tooltips (“help”) – Instead of authoring a help file (too much work), I created lots of tooltips for the various buttons, etc. So, you kind of have to explore at first and be willing to leave your mouse over a field or button to find out its purpose. The tooltips are all part of the language .ini files so one could change the tooltips themselves, if that were desired, to something that was a better explanation.
Configuration settings – There are tooltips for each of the fields/buttons in the configuration dialog so I won’t elaborate completely here. Hover your mouse over each one to discover its purpose. When you click “save”, the changes are written to the OpenSongSearch.ini file. This is a plain text file that is found in the OpenSongSearch installation directory and can be edited by hand, but for certain parameters stored there, using the configuration dialog is a safer method. One thing to note is the presence of log files.There is a log file for OSS application and one for the BaseX database. The OSS log file logs the searches that are made so that one can go back after a service and see what what searched for to see if a missing song needs to be added or if someone just typed something incorrectly. The BaseX database log is most helpful when the BaseX database build/update process fails. This is usually due to a problem with the XML in a particular song file. This log will show you which file it is and give a somewhat helpful message as to why the database build/update failed. The BaseX database log will be there regardless of whether you check the option to log the query text or not. This option is specifically to show the full XML query text sent to the database associated with the search text that was typed in. This can be helpful in debugging or perhaps in learning how the queries can be built. Leaving it unchecked does make the log growth a bit slower. At this point, there is not an option to not log or auto-maintain the BaseX log. It just grows. It’s not at all a huge thing, but certainly over time it could be. The OSS log can be recreated at each execution but there is also no option currently to turn off this log either.The button to restore default window dimensions can be helpful if for some reason the window dimensions or screen position have gotten out of whack. You may notice that on resizing the main window that relative button positions are not exactly proportionally maintained. If you close OSS and reopen it after resizing it, the buttons will be positioned in a correct manner.The “send key speed” setting can be helpful in tuning how OSS works on slower computers when it sends keystrokes to OpenSong (which is its method of interfacing with OpenSong). This may take some trial and error adjustment to find the happy medium. The initial default might be a bit slow on newer computers.
Database Update – It’s important to realize that OSS searches a copy of the OpenSong song files. The advantage of this version of OSS is that it doesn’t have to do any reformatting to stuff data into a relational database (which takes some time if you have LOTS of songs, like we do). BaseX is an XML database so OSS justs tells it which folder is the root of the song file hierarchy and it simply reads the files directly and loads them into its database…and it does it pretty fast!So, if you make a change to a song, add some songs, delete some songs, etc. in OpenSong, those changes will only be reflected in the OSS database if you click the button to launch the database update. A database of 6500 songs on my slow computer takes a minute and a half (which is a COMPLETE rebuild).Another important thing to realize is that OpenSong and OSS (or more accurately BaseX) are both XML apps and as such expect a certain “correctness” to the XML they process. Unfortunately, they don’t seem to hiccup always on the same things. When, during a database (re)build, BaseX encounters an error in the XML, the database (re)build will fail. OSS will give you a message stating that and asking if you want to view the BaseX log file (different from the OSS log file). I would highly encourage you to say “yes.” Go to the bottom of the BaseX log file and you’ll likely find a message stating what BaseX had a problem with. You’ll also get song count of “-1” if the database (re)build fails. You’ll have to correct the file in question and start again. This is very unlikely to happen if all your song changes/additions come from within OpenSong. OpenSong will write out correct XML. Usually where I see it is when I’m importing data from another format and my reformatting process didn’t do it 100% correctly. Anyway, the BaseX log is your friend when you’re importing lots of songs into OpenSong.
Searching – Now we come to the heart of the matter…actually searching for songs! The basics are that you type the text you’re searching for into the search field and click on the search button and OSS takes care of the rest. But it might be good to know some of the ways in which OSS behaves to optimize your searching:Phrases – OSS searches by contiguous text. So that means it doesn’t have the idea of atomic search elements and logical operators. It searches a single word or phrases. So, “God is great” will not find the same text as “great is God”. If you have 2 spaces between words where you should have had 1, the text will not be found (a good reason to do some data scrubbing on your song files!) Hint: don’t type in “obvious” text from the song that is likely to be found in dozens of songs. Choose a more obscure phrase and you’ll get a faster result with less hits to wade through.Accents – On the other hand, accents on characters are ignored such that you can type with or without them or even give incorrect accents and the text will be found. This has been very helpful for our French text.MRU List – The most recently used list will save the last 10 searches you type in. It doesn’t recall them as you type, but you can click on the list drop-down arrow and re-select a previously used search.The HOME key – The HOME key will bring you back to the search box and highlight the contents of the search field. This means you can hit the HOME key and then just start typing. Very convenient, so stop backspacing to erase!The ENTER key – The ENTER key will also launch the search (as well as clicking on the search button).Which fields to search on – The check boxes between the search field and the search results indicate which fields should be searched. Check the boxes that make sense for your song database and for the context of your searching. OSS can be used to help organize and clean your song database with this kind of searching.
Working with search results – Here are the ways in which you can use the search results:Sorting the results – Clicking on the title of the column will sort the results by that column. Click again and it sorts in the opposite direction.Right click – This brings up a context menu where you can choose to either view the lyrics of the song, edit the song file (I recommend that you specify a good editor that deals with XML somewhat intelligently. I personally think Notepad++ is ideal for this application.), or indicate that you want to send this song to OpenSong. Editing the song file can be helpful while you’re projecting if you see an error and want to correct it before you forget what it was. It will not change what’s on the projection screen, but it can be quite handy to deal with issues before they accumulate.ENTER key – Sends the song whose row is high-lighted to OpenSongDouble click – Sends the song whose row you double-click to OpenSong
Interfacing with OpenSong – Interfacing with OpenSong is done via sending key strokes to the song search dialog that OpenSong presents in List Mode, when you want to add an existing song to a set list or in Presentation Mode when you want to add an existing song to a presentation list. So, for OSS to work with OpenSong, OpenSong must be either in List Mode with an existing list chosen (the list can be empty or not) or in Presentation Mode. If it’s not, you’ll receive a message stating that OpenSong is not ready to interface with OSS.Depending on the complexity of the song file folder hierarchy, the actual “typing” that OSS does can be fairly rapid or a bit slow. Also, this is where the key send delay time is important to set based on the speed of your computer. If it sends key strokes too fast, then OSS and/or OpenSong can get confused. If it sends them too slowly…well, what’s the point then! ;-)If your screen real estate permits it, I suggest putting OSS above OpenSong and letting them both be visible when creating your set list and during your presentation. This facilitates quick searches without having to fumble for hidden windows.
Interface language – OSS comes with English and French translations (SVP mes amis français, ne me persécutez pas pour la traduction. Au lieu de ça, merci de m’envoyer des corrections!). Additional languages can be added by taking the OSS_LANG_EN.ini file, copying it and renaming it to OSS_LANG_XX.ini where “XX” represents the language that you’re translating into. Then simply edit the file and translate the text to the right of the “=” (equals sign) on each line into your target language. For that matter you can change the English text as well to whatever works for you. If you do create a translation, please send it to me and I’ll package it with the download.

top

Screen-shots

top

FAQ

As questions become more frequent, we’ll create an FAQ either here or in the forum. Thanks!

top

Download

OpenSongSearch.zip (7.3Mb) Current Version 2.2 – September 14, 2012

Version 2.2 changes:

Re-wrote the readme.txt file and added a LICENSE.txt file since I’ve put OSS out under the GNU General Public License
ADD – Allow paths in the configuration dialog to use standard system variables (Caleb’s suggestion)
ADD – Handle the situation of the Song Lookup dialog being already displayed
ADD – Change the song lookup dialog interaction to use the f4 drop-down and to avoid the (all) folder
ADD – Change the song lookup dialog interaction so that it can deal with a long or short wait for response from the dialog
ADD – Allow the Song Lookup dialog to be already open in presentation mode
ADD – Sped up the list view clearing
ADD – Added code in the database load to handle mixed case XML tags (to make them all lower case) because the query text was case sensitive
FIX – changed some input edits that were operating against the wrong variables
FIX – Streamlined a few bits of code
FIX – Change the message of database size when language is changed
FIX – fix the mix up in rows when you sort and then select a song to display its lyrics
FIX – Handle html entities so that they display correctly when found

Version 2.1 changes:

Added – In list prep mode, after having hit “ok”, return the focus to OSS for more rapid song adding
Added – Compile with obfuscator striponly to facilitate line number debugging in the compiled code
Added -Changed the message about starting OpenSong to reflect that the problem could be simply a language interface mismatch
Fix – Make sure the OS search dialog is active for all key stroke sending in both presentation and list mode. (fixes (subscript used with non-array variable”)