Dear IBM

As Yom Kippur starts for me here in Israel in 2 hours I want to publicly ask IBM, well everyone who reads this blog and knows me in one way or another, for forgiveness for all the things I may have said, or did, during the last year that just made you want to nsd -kill me.

A few things come to mind:

• Calling in a PMR as a Severity 1 when it probably was a 2.(I only did that once last year...I think)
• Screaming at the Fix central site for numerous reasons I will not mention but if I can ask one question, why can't I just enter my customer number to get a file? hmm?
• To all the Product Managers I have loved and lost and found, thanks for always answering the questions, no matter how difficult, even if it was to say it will be in a future release. 
• Whomever manages the certifications and virtual badges I only have 1 question, why doesn't it come in Yellow any more? Otherwise I know it is not your fault, you are just following orders.
• The QA teams. You must exist, I hope you exist, but sometimes I have my doubts. I apologize for the voodoo dolls.
• The naming committee members, oy, I am looking at a small quota on my mail file for those thoughts aren't I?
• PartnerWorld, I apologize for screaming at the most recent changes and it will not happen again...until next year's redesign if you remove my saved links again.
• Watson Workspace team for not creating a 32bit client and driving me crazy trying to get it to work, sorry about all the IMs.
• The NWTL team because I kept losing my links and having the patience to answer me 5 times where to upload the files. Sorry did not get to do it again this term.
• The content management teams that asked for guest blog posts, I find it very hard to write for a non customer facing product so I apologize for anthropomorphisizing Websphere. But #DominoRocks via IBM Notes.
• HCL I am really sorry I could not use my Golden Ticket. #Domino2025 will be here soon and I look forward to what you bring us.

I'd also like to apologize for the number of times a year I say, Quickr would work so well for this project.

Mission Accepted: Lotus Quickr to IBM Connections Cloud

My last Lotus Quickr customer is moving off of Quickr by December 31st.

I love Quickr, still do, but time marches on and maybe you found yourself looking at Quickr wondering:

• How do I secure this box? You have options, and hopefully have used them, but the server is trapped on an 8.5 server which is basically an unsafe server to run at t his point especially if it has access to/from the outside world.
• Accessing your data is getting harder as browsers advance but the code on the server does not recognize all the updates of course.
• Mobile needs, never really got a working Mobile client for Quickr
• Java, yeah well it was good while it lasted

The thing is, if you are solely using it internally it can just sit there for years on end, some of you out there know exactly what i am talking about too.

IBM does not provide a tool to migrate data from Quickr to Connections. IBM Global Services relies on a tool from a Business Partner and you can go that route. It is priced accordingly.

However, in this case my client is not that large and only has about 300GB of data in Quickr so we decided to move it using the tools available to us.

I did many tests before it was agreed to move to IBM Connections Cloud, the client already owns and uses Mail and Sametime there so the move to it would not involve new licensing. 

In fact, we saved money by not needing to renew our Lotus Quickr, now called IBM Connections licenses and support.

Once it was shown we had viable functionality, which basically meant many tiers of FOLDERS that could be created and edited via Notes, Plugins to Windows and MACs, we were ready to go. 

Sorry taxonomy and search people, but when you need many files quickly, search is not your friend especially if you have no idea what the file is called. This is a religious argument to have at the next LUG or IBM Connect event.

So if I didn't go with IBM, how did I do it? Same way I have done migrations for over 20 years, ask around and build and leverage what you have to work with and it should save the client about 40% of the Global services price. 

Your mileage may vary, but if you are in need of this, my January is free now so call, email, tweet, skype me to help move you to IBM Connections Cloud from Quickr.

Next post will be about the process involved, for those interested let me know so I can provide more details to help you.

Paid or Free, Support by Vendors is lacking lately

No idea why but lately support services at 2 major vendors are really slow to respond.

In one case Symantec has been working on a backup issue, we can't get BackupExec to "see" the Domino directories to select them for backup on a primary mail server at a client. This has been dragging on for weeks. Along the way we resolved some other minor issues with other servers, but this is a main mail server. The files are of course replicated elsewhere, but still, weeks? I go days without hearing from them, then I get like 2 days of emails and calls and more log files FTP'd.

Because of this we are looking at some other vendors, Appassure looks promising, but they have no idea how Domino will work with them, I am a first client. Will get to them May 1 or so.

Another case involves a vendor which promises support for their Cloud offerings. They do and replied quickly, but have yet to reply to my replies asking for an email or account to share the file in question. Hard to discuss an issue if support can not look at the files. I emailed it to them finally in frustration only to get some email about my using FF4 and Windows7, or the 2 together was not approved. Well, sorry but I am testing for a client and that is what they use. Also was told the file should not have advanced formulas in it.

Please define advanced formulas? Is HTML advanced? Colors? Math calculations? This one left me scratching my head a bit.

So the Cloud is getting people to use it, but if it is less useful than an iPad or laptop or even my phone, what is the point? Doesn't everyone have file servers and shared location sites for files? Maybe Google Apps really is that good? Client wants to look at everyone's Cloud offerings now, MS, IBM, Google, you name it I guess stay away from local Cloud vendors?

All in all not a productive use of my time but since these are deep issues for clients I deal with them because my creative mind and troubleshooting may fix it when or if I get a support person to talk to me.

IBM Support has changed, have you?

Keep in mind my original posting about how to work with Support.

I, like many of you, received an email(well at least 4 of them) saying that I will no longer have to endure waiting on hold when calling support.

They will now put remote technical software support for Lotus products (with the exception of Lotus Foundations and LotusLive) provided exclusively in call back mode and we get to wait for someone to reply to our request. Starting Monday February 16, 2009.

Huh?

Rarely do I ever wait on the phone when I call in with an issue and usually they let me know if it is in call back mode already and I am ok with it, unless it's a major server failure issue, category 1 and even then the wait is short. I must be one of the lucky ones I guess since I rarely sit on hold.

You will still be able to open support requests electronically as well via the Web-based Electronic Support Request (ESR/SR) tool.

By the way in at least one email the URL was incorrect for the ESR tool, it should be this http://www-01.ibm.com/software/support/probsub.html, the link above is corrected.

Great news and when I have been in a situation with Internet access and a browser not locked down it has worked well, BUT the next line says:

our new Web-based tool that will be extended to handle ALL software related electronic service requests over the next two years.

Hopefully all of the products we support are on there already.

Further along...

Our customers tell us that increased access to information and the ability to be self-sufficient in finding the solutions they need, when they need them provides the optimum capability to manage their IT infrastructure.

Yes, and we do appreciate everything IBM/Lotus can offer in this area, but if I am calling them it really is because I have already done the following (covered in my speedgeeking session, slides coming):

1) Wracked my brain of 15+ years of personal Lotus support knowledge and that of my coworkers
2) Searched with the IBM Support Toolbar(get it NOW! for Firefox or IE)
3) IBM Support Assistant
4) IBM Support Forums (and BP Forum), you can go direct to the forums from the IBM Support Toolbar
5) Lotus Self-Assist Support Site
6) Fellow Bloggers posts, usually searched from Planetlotus.org
7) Twitter / Facebook posts/questions
8) Bleedyellow.com pinging on Sametime other admin buddies
9) Pinging on Sametime IBM internal people I know if it's their area of specialty
10) Emailing other people I can't find online or on IM or any of the above
11) Posting to Linkedin Forums, like Lotus Software Professionals
12) Posting to l-Notes newslist or other similar ones
13) Google as a last resort, thus lucky #13

Edited 3/29/2009: 14) Brian's notes.ini add-on for Firefox and IE7
So there are at least 13 ways I can track down a problem, more if I use all the IBM self assist routes(the support toolbar covers 90% of them), before I call into Support.

So who has spent the time wisely or more efficiently? Now I must wait longer to get a reply? Hopefully not and this is just a formality to hand over a call to someone who will return my call within a few minutes.

Just remember when you call or get your call back, the people on the other end are just like you and me and have their own internal issues and deadlines and quotas and please try to work with them not against them. It makes everyone's life easier.

SnTT - Waiter there's a local ID lost on my server

Ripped from the headlines....well maybe one of my clients servers, I am happy to bring something to light which happens, sometimes, for no reason, that my client can say, yet I must fix it anyway.

Good thing I am not hands on anymore these days? Riiiiight! Because 15 years of Domino Admin experience gets me all the problem clients no matter what I am doing these days.

So back to why I asked you here today to read my minds thoughts.

The server console spits out, every hour consistently mind you the following:

Error updating local ID file: The public keys specified in the Name Change Request do not match those specified in the new certificate

Now raise your hand if you haven't seen a "similar" message before. It happens sometimes, in fact there are tons of technotes that start with "Error updating local ID file" but only 1-2 that include my error.

Now remember in my previous post I said do at least 5 things before calling support, well I did the following:
1) Immediately checked the server ID file and it's date of expiration, domain, certifier, etc.. All checked out ok.
2) Went to the Admin4.nsf databsse, otherwise known as AdminP's database. Oddly enough no reference to name changes, certifier changes or anything even remotely close to the name of the server at issue,
3) Next up Certlog.nsf, just checking expirations maybe someone inside decided to make sure the server's wouldn't expire in 2080 as designed. Nothing unusual, although a number of people's ID's are going to expire this year, made note, informed client just to check.
4) When specifics don't work, go back a step to more general areas like the Server Doc. No why would I do that for a name change you ask? Simple, what if someone decided to textually change the name without recertifying? Guess what server isn't going to talk to anyone? While all looked okay I noticed there was no entry under port in the Ports tab. Odd I thought didn't notice anything funny in the Admin4.nsf database for errors. Should go check..
5) The server's log.nsf. And what do I see, an admin process error every time the server was restarted. Swap over to the administration server, tell adminp to process all and notice errors and some other pieces getting done. Adminp must have been turned off or stuck.

Now replicate changes to the server in question and the ports field is now fine. I then check on security and some other fields I like to check and find the lookup to the NAB fails. Errors. So, check it out and on the server in question, the NAB shows a number of replication conflicts. Fixed them, deleted them, ran a quick CTRL-SHFT-F9 and restarted the server. No more admin process error but local ID error is still there.

This Technote, #1097801 explains how to resolve the problem.

Now for you newbies out there. It should scare you to do what the technote says.
I hate playing with public and private key information unless I really must do it. So many ways to mess this up and render your server DOA if you are not careful.

That is when I called IBM. Level 2, because playing with certificates is NOT something to be taken lightly. Luckily Geno was able to work with me on it.
We ran some debugging (set config debug_threadid=1) first to see if the error was from an agent or an application, you run (sh ta debug) but it didn't show us anything aside from it being a server process.

Went over many choices and ideas and we each searched our respective knowledgebases and came up with the Technote above. Well when all else fails, you try it.
So follow the Technote steps in simple are:

First make a backup of the certificate public key you are deleting from the server doc, just in case.
From the administrative client of the server:
1. Select File, Tools, UserID (assuming the client is using the same ID).
2. Select More Options.
3. Click Copy Public Key.
4. Open the Server document, delete the existing key from the Certified Public Key field and then paste the newly copied key into the field.
5. Save the document and restart the server

And it worked. Another client happy.

To turn off the debug code change the =1 to 0.

Plus I learned that Geno follows me on Twitter(Edited jan2009 with his permission) and reads my posts. As much as this is nice to know, it's better for me to know someone on top of it all is there when I/we need him.

So to Geno and his boss, Mark H. thanks and give Geno an extra coffee break, answering our calls can drive a person crazy.

Lotus/IBM Support Something you should know

Note to IBM, make the Support teams have some logo, nothing on the IBM site for it that i could find. Even an orange circle with an H inside would be nice.

Many years ago they were the best when you needed help. Then like many organizations do they had some trouble spots along the way, but I am happy to say they are back in full swing and really worth their money.

But you should remember they are just like you, and you may have been one of them once, or still are in some ways, so be kind, be nice, help them too if you can and listen to their ideas, especially if you are out of them.

1st thing to know is always have IBM Support on speed dial.

2nd thing is DO NOT HARASS them until you have done at least 5 things first on your own to trouble shoot the problem, otherwise they will probably make fun of you in their weekly meetings.

In general if you can't come up with 5 things to check for any problem, I teach classes in general troubleshooting and of course specifically about Domino too when asked. Yes, pinging someone is preferable over the lazy web way of twittering for an answer. We are all here to help, just ask us.

Seriously though Lotus Support is very good these days and better at raising a flag to the developers if necessary so go ahead try them again. I am sure they don't make fun of you, well, okay, maybe some of you, even me!. You know sometimes your brain goes fuzzy, too many things on your plate, the game is on in 45 minutes, significant other is ....you know we all have reasons that we need an answer NOW.

Here are basic guidelines:
NEVER, EVER call and say it's a Level 1 unless your server is really DOA, finished, game over. It's just not fair to scream fire all the time.

Use level 2 for anything else that is amiss or odd, except questions.

Questions of general or what if or how about are Level 3. For instance does R8 accept importing from Excel instead of a wk3 spreadsheet. No it doesn't, stop bothering them.

And with that this PSA is over and I can go back to building demo's for Lotusphere.