How does 1.2M emails sound?
There are people out there who say HCL Domino can't handle the stress of modern times.
It is an old system(truth be told, Exchange isn't much younger) with limitations.
Well, yesterday, I did an unintended stress test of Domino and a client's internal infrastructure.
Like most large and well-known companies, they run many Domino applications that handle millions of dollars a day but also have an O365 infrastructure.
Mail doesn't come into Domino, but it does go out from there, and this is where it got interesting.
Domino sent over 275,000 emails over about 15 minutes +/-.
This was all internal SMTP, so we didn't get spam blocked or anything,
Normally, a company would have a choke hold option on mail sent per minute, but for some reason, that was not in place or was avoided due to routing configurations yet to be determined.
And my 275K grew to 1.2M inside the network architecture.
Fun day at the office, right?
No, I caused the problem and also knew how to fix it, but that small window of time was enough to unleash the tsunami.
Sometimes, the simplest things become the hardest things, and this was such a case.
Even veterans of the email wars screw up.
I now have a great session for F*uck Up Nights if anyone is interested.
Definitely used up one of my IT 9 lives, this brings me down to about 5 left.
HCL Domino was amazing throughout. Not only did the servers not complain about anything, but they just kept going and going and going. Multithreading FTW!
O365/Exchange did not do so well, got clogged up and had issues because it still is not a multithreaded service.
I pulled the numbers from the Mail statistics and showed them on my Admin client Monitoring dashboard. The left side is tasks, which I covered in my previous blog post. On the right side, you can add statistics, something not many people do, but I like to see some things there and probably should do a separate session on the statistics, but that is for another time.
My peers, I am sure, can guess what caused the tsunami, so there is no reason to elaborate. But let's just say when you are a junior admin, this is one of the outcomes of your trial-by-fire Domino Administration education.
For those who think Ambassadors and long-time Yellowbleeders are these great Gods of tech, some really are titans, I admit my mistake and that, indeed, you are never too young or old to learn something new....or mess up royally.
No comments:
Post a Comment