What can IBM do to make Domino development better?

In a recent blog post Ed Brill poses the following question:

In a world where the majority of new applications are being built in Java (and associated languages) and .NET, what approach could IBM use today to attract hoardes of new developers to learn Lotusscript? Isn't this the whole reason that Notes get labelled "proprietary" (in the tone of voice where proprietary = bad)?

"Hannover" will deliver the best of both in that the existing model will continue and be enhanced, yet the client will be open to new Java-based development methods. Thus the inflection point I mentioned earlier.

I don't want to suggest that there's no new Notes developers or development going on -- I get to hear about cool new projects every day -- but clearly it's a different market dynamic for developers today than it was when Notes became mainstream.

It's a tough question, and not one that can be answered without being a bit harsh. The short answer is you can't attract new LotusScript developers. It's a limited-use language with no future. LotusScript is a one trick pony, if you use it you're locking yourself into doing only Notes development, and the future of Notes by no means guaranteed. Using a general-purpose language such as Java or C# gives the developer more marketability.

The other major issue is Domino Designer itself. The IDE lacks features people have taken for granted in other development tools since the early 90's. LotusScript debugging is rudimentary and Formula, Javascript and Java debugging is nonexistent. New features have been tacked on over the past seven or eight years in ways that make them difficult to use. HTML rendering, which was introduced in R4, is STILL messed up. CSS support is spotty at best. A lot of this is supposedly being addressed with Hannover, but I'm not holding my breath. Nearly a decade of promises that it'll get better has left me jaded, and it has run off a number of people who might otherwise have embraced Notes and Domino.

It seems that IBM is forcing Notes developers into Java. That's fine, it's IBM's decision to make. I find it disheartening, but not unexpected. Notes and Domino are at critical inflection points. I appreciate that IBM and Lotus are trying to walk the fine line of delivering future-focused products while not alienating their existing customer base. I also appreciate people such as Ed asking these questions and not shying away from the scrutiny.

SNTT - How to create a Windows shortcut from LotusScript using Windows Scripting Host calls

Recently I had a need to deploy a Microsoft Access application to a number of users. I needed to give them each their own copy of the database and give them a shortcut to it. There are a number of ways to achieve this, but I decided the easiest would be to just use some Windows Scripting Host calls. Here is a short snippet of code that will get the location of the currently logged in user's My Documents and Desktop folders, copy a file from a network drive (P in this case) to the My Documents, then create a shortcut to it on the user's desktop.

Dim sDesktop As String
Dim sMyDocs As String

Dim oShell As Variant
Dim oShortcut As Variant
Dim oFS As Variant

Set oShell = CreateObject("WScript.Shell")
Set oFS = CreateObject("Scripting.FileSystemObject")

sDesktop = oShell.SpecialFolders("Desktop")
sMyDocs = oShell.SpecialFolders("MyDocuments")

'Copy the file
Filecopy "P:\Charles\BPS Barcodes.mdb", sMyDocs & "\BPS Barcodes.mdb"

'Create shortcut
Set oShortcut = oShell.CreateShortcut(sDesktop & "\BPS Barcodes.lnk")
oShortcut.TargetPath = sMyDocs & "\BPS Barcodes.mdb"
oShortcut.Save

Messagebox "Installation successful.", 0, "BPS Barcodes"

This LotusScript was converted to HTML using the ls2html routine,
provided by Julian Robichaux at nsftools.com.

SNTT , show-n-tell thursday

SNTT - Notes Community Template Project - Enough of the talk, let's do this

John Head made another posting about the community effort toward providing more and better templates for Notes. The reason we're having this discussion is because IBM is unwilling to invest in this idea and has suggested the Notes community pick up the banner. Bruce Elgort suggested we take the top 20 downloaded templates from OpenNTF and spruce them up. Others have suggested we work from the list of templates from Sharepoint. Since no one has taken the lead I am stepping up. My commitment is to:

• Provide consistent Notes client UI documentation with recommended best practices based on the Notes 7.01 mail template. I don't do web development and would appreciate it if someone could do something similar for web applications. I will be posting my UI guidelines on or about August 26th.
  
  
• Provide a framework for deploying these guidelines. The most likely vector for this is the excellent work of the openslice project.
  
  
• Provide a best practices document for Notes development that discusses template-driven Notes development.
  
  
• Provide nifty modules of code that encapsulate common Notes application functionality.
  
  
• Work with any interested OpenNTF contributors who would like to apply these guidelines and framework to their own applications.
  

I am interested in participating in any larger movement that gets started and do not intend to cause conflict or discord. My intent is to start laying the considerable groundwork that will be needed regardless of the final direction this community effort takes.

SNTT, show-n-tell thursday

Vacation time and happy birthday

I'm leaving on Saturday, August 12th for a well-earned vacation. My partner and I plus four friends are going to Tortola, in the British Virgin Islands. My partner and I traded a few extra timeshare weeks for a three cabins on a catamaran. We're returning on Saturday, August 19th, which is my birthday.

So no blogging or responding to blogs for the next week. Enjoy the silence.

Side projects in the works

I've taken on a rather ambitious set of side projects that I'll work on as my time and interest permit. If anyone has any strong opinion about which one they would like me to complete first let me know. I'll be sharing source code and writing some how-to articles as I go through each one. The following are in no particular order.

• SQL Query Runner - I spend about 95% of my time digging for data in our ERP system and executing queries to see if the descriptions they provide match the data in the table. I have been using a crippleware utility that will only execute queries that run in under 15 seconds. I'm a developer, I can fix this. I'm going to write my own query execution tool and do it in C#. I end up with a tool I can use and I learn a bit more about C#.
  
  
• DXL UI Skinner - A core issue that keeps resurfacing in recent blog buzz is about how difficult it can be to deliver a consistent UI across Notes applications. Templates and UI guidelines are a great start, but the first time you customize a Notes UI element you can no longer synchronize it with a template. Sure you could create a separate template for every design element, but who really has the time for that? My idea is to use DXL to apply UI updates to customized design elements. I know it's been done before but I haven't found a utility that will do it, so once I get mine done I'll put it up on OpenNTF. First up will be form and view action bars and view column headers. If there are any specific design elements or properties you'd like to be able to update en masse let me know.
  
  
• Barracuda Spam Statistics - We use a Barracuda Networks Spam Firewall 300 to filter inbound mail. It works extremely well, we block 85% of all inbound mail with very few legitimate false positives. (Users think the chain letters from their friends are legitimate, I take a different view.) It generates statistics and sends reports via e-mail, which I route to a mail-in database, parse through them, and collect statistics so I can have some idea of how much spam we're getting. Right now I'm only collecting summary traffic data. I will be expanding this to include the rest of the Barracuda reports: top spam senders, top spam recipients, top mail recipients, and top viruses. The Barracuda also exposes a CGI interface for getting access to more information, so the next step after I get the reports parsed will be to research the CGI options.
  
  
• DNS server - Believe it or not, where I work we do not have an internal DNS server. Now that I'm getting more comfortable with linux I'm going to dive in and set one up. I only need to serve two domains and it will only be internal traffic, so I don't anticipate it being too difficult.

SNTT - iSeries to RHEL migration - Getting Domino Web Access to work

As I previously posted, moving Domino from one server platform to another is a relatively simple process: just install the same version of Domino then copy the entire Domino data directory from the old server to the new one. When you do this from an iSeries to RHEL you have to take into consideration that linux is case sensitive while the iSeries largely is not.

I ran into this after I moved my mail over and Domino Web Access wasn't working. After extensive troubleshooting I finally posted on Notes.Net and a helpful person there jogged my memory that led me down a path that yielded a resolution. A number of directories on the iSeries are in all upper case, but RHEL expects them to be lower case. Here's how I fixed it:

1. From the Domino console
   tell http quit
   
   
2. From the linux console (I was logged in as root but your notes user should work)
   cp -r /local/mail/domino/JAVA /local/mail/domino/java
   cp -r /local/mail/domino/TEMPLATES /local/mail/domino/templates
   cp -r /local/mail/domino/PLUGINS /local/mail/domino/plugins
   cp -r /local/mail/INOTES /local/mail/iNotes
   
   chown -R notes:notes /local/mail (not needed if you're logged in as notes)
   
   rm -rf /local/mail/domino/JAVA
   rm -rf /local/mail/domino/TEMPLATES
   rm -rf /local/mail/domino/PLUGINS
   rm -rf /local/mail/iNotes
   
   
3. From the Domino console
   load http
   

In step 2 above you could use the mv command instead. I prefer to do it in two steps, copying and testing that it worked, then doing the remove separately.

SNTT

Why I'm migrating from an iSeries to Red Hat

I've taken a few lumps recently about my decision to move from Domino on an iSeries to Red Hat Enterprise Linux (RHEL). My reasons for the migration are as follows:

• It is incredibly expensive. Just getting a baseline iSeries with a single CPU, 2GB RAM and 150GB storage is going to run you over $20K. In my case upgrading from 1GB RAM to 4GB and 5 drives to 8 drives was going to cost over $10K. This was in addition to the $40K we had already paid for the box.
  
  
• OS/400 an arcane beast and finding good information is extremely difficult. I have a cursory understanding of how it fits together, but after eight years working with it I'm still mostly clueless! It just runs, which is good, but when there are problems I haven't the foggiest idea where to go to figure out what is going on. I end up spending lots of time on the phone with IBM support. They're good, but I would really appreciate something a little more approachable with better documentation.
  
  
• Upgrading was a pain. Each release of Domino came with it a list of the patches you had to have on your iSeries, as well as what version of OS/400 you needed. Verifying the patches was a tedious process, installing new ones took even longer, and heaven forbid you actually had to get an OS upgrade to install Domino. Domino 7 requires i5/OS 5.3, which our iSeries won't support, so we could not upgrade to Domino 7.
  
  
• IBM doesn't seem to know what to do with the AS/400 / iSeries / System i5. They keep renaming, rebranding and remarketing it. The latest push has been to use the iSeries to run linux. I just don't see that IBM is really moving the platform forward.
  

A couple of things to keep in mind regarding my perceptions. First, I have taken no AS/400 or iSeries classes. That would address my lack of knowledge, but not the rest of the issues I described. Second, I don't have an IBM or Lotus business partner I work with regularly. I have worked with a few in the past and feel that I received incorrect or even downright wrong information from them. In my most recent experience purchasing a BladeCenter and IBM SAN solution the BP was basically a mouthpiece so I could get discounted pricing. Third, I don't speak to any IBM or Lotus representatives on a regular basis, unless you count Ed's and Alan's blogs.

The decision for the migration has been made, the new hardware has been purchased, and I'm in the process of doing my migration. This is just an explanation of the reasoning that led to the decision.

The Notes community's involvement

Recently Ed Brill posted an article about a prominent Australian company dumping Domino in favor of Exchange. This sparked a disucssion which led to a lot of people saying the problem was Notes didn't ship with enough canned templates.

Personally I've never used any Notes templates other than the standard ones for administration databases. I created exactly one each of discussion databases, document libraries, team rooms, and database libraries. I tried to wade through the code, found it more confusing than helpful, and went off on my own... and never looked back. In my seven years developing Notes applications I have shared databases with a colleague who works in the same industry, but I have not ever used anything from OpenNTF, the Sandbox or any of the updated templates provided in newer releases of Notes.

I'm telling you all that to give you some background in where I'm going with this. John Head took up the torch from Ed's blog posting and suggested that maybe it's time for the community to step up and fill the need for more templates. On the surface that's a great idea.

The early dialog centered around duplicating the templates provided by Microsoft as part of Sharepoint Portal. My concern was that this was headed in the direction of serving as nothing more than a marketing tool for business partners to sell more of their own services. The templates would be little more than a slick UI with little behind it, leading to my analogy of a Potemkin's village of templates.

For me to get fully on board with this it has to provide fully functional applications, not just proofs of concept. In this vein Bruce Elgort suggested we take the top 20 downloaded OpenNTF templates and bring them up to speed. I think that's a fantastic idea, and I'm on board with it. It's certainly more productive than endless hours of Guild Wars. :-)

SNTT - the Win32 API series - FindWindow and LotusScript

FindWindow is a Windows API call that gives you the handle to a specific window. For example, you can use FindWindow to get the instance of Excel then bring it to the front, resize it, close it, or any number of other things. A window handle is a good thing to be able to get.

The typical declare statement for FindWindow looks like this:

Declare Function FindWindow Lib "user32" Alias "FindWindowA" (ByVal lpClassName As String, ByVal lpWindowName As String) As Long

The first parameter is the class of the window you want to get. For Notes it's just "Notes", but for Microsoft applications things get a little crazy. If you don't care which instance of an application you get you can just pass a placeholder for the parameter you don't care about. For example, to get the Notes window from VB6 or prior you would call it like this:

hWnd = FindWindow("Notes", vbNullString)

vbNullString is a constant that evaluates to "", so I thought I could simply use the following from LotusScript:

hWnd = FindWindow("Notes", "")

That doesn't work. After extensive searching, and help from friends in the Efnet IRC channel C#, I learned that the VB runtime is actually passing 0 when you use vbNullString. There's a lot going on with marshalling and LPSTR and other stuff I don't fully understand, but the upshot is that to get this working from Notes you have to change the declaration:

Declare Function FindWindow Lib "user32" Alias "FindWindowA" (ByVal lpClassName As String, ByVal lpWindowName As Long) As Long

And then you can do:

hWnd = FindWindow("Notes", 0)

Okay, so if you know the class that's great, but finding the class can be a bit of a pain. What if you do know the exact window title? Then you have to declare it with the class as Long and the window name as String:

Declare Function FindWindowByClass Lib "user32" Alias "FindWindowA" (ByVal lpClassName As String, ByVal lpWindowName As Long) As Long
Declare Function FindWindowByTitle Lib "user32" Alias "FindWindowA" (ByVal lpClassName As Long, ByVal lpWindowName As String) As Long

Crazy, I know. I'm not sure if the fault is LotusScript's inability to pass pointers or Microsoft's API implementation. But this should help anyone trying to use FindWindow from LotusScript.

SnTT

Migrating from iSeries to Red Hat - Part 1

Moving Domino from to a new hardware and OS platform takes careful planning and lots of testing. I have been working on this project for over a year and wanted to share the lessons I have learned along the way.

Currently I run five Domino partitions on an iSeries 270 Dedicated for Domino. Yes, that's one of the bumblebee boxes. I'm migrating to an IBM BladeCenter with a DS4300 SAN. We have two blades for Domino, with mail services on one and applications on another. I am debating installing a secondary Domino partition on each blade and using them for clustering, but that's another set of posts.

The first step in this process was to determine the best approach for moving things over. After extensive research and lots of testing with VMWare Workstation I decided on the following scenario:

1. Install the same version of Domino on RHEL that I have on the iSeries. In my case this is 6.5.4 FP3.
   
2. Create a folder in /local/ for each Domino partition
   
3. Create an IFS share on the iSeries for the root Domino folder (all my Domino partitions are installed in /NOTES/ on the iSeries)
4. Mount the share on linux and copy the files from the iSeries to linux
5. Adjust notes.ini to use the new path structure
   

Many people move Domino to new hardware in conjunction with an upgrade to a new version. Don't install a new version of Domino on the new hardware and try to copy everything over. That is difficult to do correctly unless you really know what you're doing. Using the two-step approach gives you the opportunity to create a known working installation before you do the upgrade to a new version.

That's the 50,000 foot view of where I'm headed. In upcoming posts I will cover installing RHEL and configuring it for Domino, installing Domino, configuring ODBCDirect database drivers, migrating servers, and tweaking applications so they work correctly. I expect to learn a lot throughout this process.

SNTT - How to shell a Windows application from LotusScript and close it

A recent post on Notes.Net asked about shelling a Windows application and closing it. I suggested the poster look up a couple of Windows API calls that would help, then I went digging for them myself. It wasn't quite as simple as I expected to find the necessary information, so I thought I would post it here:

(Declarations)
Const SEE_MASK_NOCLOSEPROCESS = &H40
Const SEE_MASK_FLAG_NO_UI = &H400

Private Type SHELLEXECUTEINFO
cbSize As Long
fMask As Long
hwnd As Long
lpVerb As String
lpFile As String
lpParameters As String
lpDirectory As String
nShow As Long
hInstApp As Long
lpIDList As Long
lpClass As String
hkeyClass As Long
dwHotKey As Long
hIcon As Long
hProcess As Long
End Type

Declare Function ShellExecuteEx Lib "shell32.dll"_
Alias "ShellExecuteEx" (SEI As SHELLEXECUTEINFO) As Long

Declare Function TerminateProcess Lib "kernel32"_
Alias "TerminateProcess" (Byval hProcess As Long, Byval uExitCode As Long) As Long

Sub Initialize
Dim SEI As SHELLEXECUTEINFO

SEI.cbSize = Len(SEI)
SEI.fMask =  SEE_MASK_NOCLOSEPROCESS Or SEE_MASK_FLAG_NO_UI
SEI.lpVerb = "open"
SEI.lpFile = "c:\sample.txt"
SEI.nShow = 1
SEI.hInstApp = 0
SEI.lpIDList = 0

Call ShellExecuteEx(SEI)

Messagebox "Application opened.", 0, "Success"
Call TerminateProcess(SEI.hProcess, 0)

End Sub

This LotusScript was converted to HTML using the ls2html routine,
provided by Julian Robichaux at nsftools.com.

SNTT , show-n-tell thursday

My love/hate relationship with Domino

I have been very critical about Lotus Notes, Domino and IBM in general. Some people have taken that to mean I don't like any of the above. This posting is to set the record straight and demonstrate that is emphatically not the case.

My background is pretty broad, ranging from PBX administrator to DBA to developer. As a developer I have worked with mostly BASIC-like languages: MS Access VBA, Visual Basic, and LotusScript. I have always been intensely interested in data. How it gets collected, how to manipulate it, and how to turn it into useful information. This love affair was awakened in my first programming job, which was using MS Access 2.0.

When I tried to transition from MS Access to VB I quickly grew bored of writing data access classes. MS Access had all that built in and was easy to work with, but in VB I was rolling my own code every stinking time. (I refused to use data-bound controls in VB4/5/6.) So I ended up in a job generically titled "PC Specialist". I was "the IT guy" -- if it had a plug on it, I was responsible for it. This is where I was exposed to a vast array of technologies, responsibilities, and systems. Part of this included making the decision between Exchange and Domino in 1999.

Management wanted to use Domino because our primary supplier did. I didn't know much about Domino or Exchange, but a few things turned me off to Exchange from the start. First, I had serious concerns about my (in)ability to properly secure Windows servers and maintain a high level of security. Second, Exchange required a sprawl of servers and services to function properly and I was intimidated by the prospect of trying to keep all the balls in the air just to get e-mail to flow. Third, programming applications meant using VB and SQL Server and going back to the old hell of writing data access code. Notes development couldn't possibly suck more than that.

And it didn't suck, it was simply amazing. Notes had most of what I loved about MS Access, that being rapid application development tightly bound to the underlying datastore, while being server-based and including granular security. I spent several years gleefully plumbing the depths of what Notes could do, getting my R5 PCLP along the way.

As my applications became more complex I started hitting walls. Debugging Lotus Connectors was difficult and frustrating, and the documentation was atrocious. New features in Domino, such as policies and remote debugging, simply did not work. Clustering Domino servers across multiple platforms required lots of workarounds that you only found by trial and error. I grew to loathe Domino Designer and it's bizarre property window, code stuffed everywhere and the half-baked classes implementation.

So off I went to find the new new thing, something that offered everything Domino had without all the annoying bits. In my quest I tried out MS Access 2003, C#, VB.Net, J2EE, and Ruby on Rails. Broadening the scope I worked with LAMP, OpenACS, Zope/Plone, and Puakma. Most recently I tried out SharePoint Portal 2007 and Exchange 2007. I never found anything that could reach the bar set by Notes and Domino.

Nothing else offered the ease of rich client development, granular security, and an integrated single-server install. There are still annoying parts about working with Notes and Domino, but in the final analysis they are just annoying. The fact remains that you cannot replace Domino with anything else without a tremendous amount of effort. I've climbed that mountain and found there is no summit. The only viable integrated collaboration platform available is Domino.

For all it's knobby warts, I truly do love Notes and Domino. I see in Hannover an opportunity for IBM to finally get it right. The criticisms I voice stem from an intense desire to silence the naysayers.

Domino migration - iSeries to Red Hat

We currently run Domino 6.5.4 FP3 on an iSeries 270. Several months ago we purchased an IBM BladeCenter with the idea of migrating the Domino servers to that. I have done lots of testing and I'm finally starting that process. I intend to blog about all the steps I take to get there, focusing heavily on the "gotchas" and things that I have to really dig to find.

Here's an overview of what I'll be doing:

• Moving Domino server named MAIL to its own blade
• Moving Domino servers named Intranet, Armstrong, and TranSouth to the same blade
• Use Internet Sites documents to service two websites from this second blade
• Use ODBC to connect to an iSeries DB2 server (not the same one currently running Domino)

On the surface it sounds relatively simple. In testing it has been a bit more difficult to get everything talking to each other. I'll be doing the MAIL move on Saturday, July 22nd. If that goes well I'll start the Intranet move, too.

what's the cubert thing about?

I've been asked by a few people why I use the name cubert. It has nothing to do with the 80's video game. My partner and I have a Pembroke Welsh Corgi. When we were researching names we decided to give her a real Welsh name. We came up with several, but many were nearly unpronounceable by us. We finally came up with Beti, which is the Welsh spelling of the English name Betty and pronounced the same way. Here is Beti as a puppy.

But that doesn't explain why I took the nickname cubert. While doing this research for Beti I stumbled across the name Cubert. In Welsh mythology he was the son of Daenre, a succubus, and an unnamed hero. I like mythology, I liked the name, so I took this as my own. :-)

LotusScript Performance Tip - Clearing the object cache

A while back I was browsing SearchDomino looking for an answer to a problem and stumbled across a posting by Andre Guirard that made a reference to using the Delete statement to remove NotesDocument objects from memory. I was intrigued, so I asked on Notes.Net and Andre responded.

The relevant code is in Andre's response, but in the interest of completeness I will repeat it here:

Dim session As New NotesSession
Dim coll As NotesDocumentCollection
Dim db As NotesDatabase
Dim docCur As NotesDocument, docNext As NotesDocument

Set db = session.CurrentDatabase
Set coll = db.UnprocessedDocuments
Set docCur = coll.GetFirstDocument

Do Until docCur Is Nothing
   Call session.UpdateProcessedDoc(docCur)
   Set docNext = coll.GetNextDocument(docCur)
   ' INSERT CODE HERE TO PROCESS A SINGLE DOCUMENT
   Delete docCur ' so documents don't accumulate in cache
   Set docCur = docNext
Loop

What I'm not clear on, and have not been able to test yet, is whether this also applies to Forall loops and not just GetFirst/GetNext constructs. I also don't typically use NotesViewEntryCollections, so I don't know if it applies to those, either.

SNTT