5 inevitable red flags to watch out for in complex database systems

The goal of any organisation, business or otherwise, is to grow.

In the past, the size of a company was determined by the number of physical products that one either created or sold. Say you built and sold left-handed toolboxes. If you sold enough of these left-handed toolboxes, and if they were of good enough quality, more people wanted your toolboxes, so you created more.

With this growth, was a need to keep a record of these products, customers, sales, etc. Companies simply kept records in pen and paper, in stacks of ledgers. Soon these started to fill filing cabinets and these cabinets started to fill rooms and even buildings.

Computers made things a lot easier; instead of paper, ledgers were stored in text files. After that came spreadsheets, which were fantastic at first (anyone remember Lotus 1-2-3?) but soon became inadequate for large amounts of data. Next, relational databases helped organise information. 

As this data became more and more complex, the economy changed to be more focused around information itself, so information storage itself was essential to even the most basic operations of any company.

However with change comes new problems.

A Mix of Different Platforms and Versions

Much like in the past, as companies and organisations grew larger, the methods storing information grew more and more complicated. As a result, that simple database that your organisation used to manage your data? It was no longer sufficient. More databases were added. As new platforms became available new functions of your business were stored differently.

The earliest business was stored in an old Access database (yes, admit it, you know that’s how you started, or maybe you can earn some real credibility and talk about your old DB2 systems to anyone who will listen… but they won’t).

So you moved to more complex systems, like Oracle or Sybase. However as new segments of your business were added, different platforms were used (or new DBAs preferred them).

Soon you ended up with a far more complex array of different systems. Even within databases running the same platform, you now have different versions, not all of which are compatible. Why not upgrade the previous ones? Several factors make this unlikely, or even a good idea. Sure you can migrate everything to a new platform but this is time-consuming, and also expensive. Also, every time some new system comes out, you’d need to do the same thing over and over again. This quite simply would slow down your entire organisation and leave your business in the dust.

Many Different Tools All With Different and Complex UI

So you are stuck with this rainbow of platforms. Of course you need to monitor and maintain each of these systems. However due to the differences between them, you may end up with any entire infrastructure of tools that you use for monitoring each of platform.

Each tool is designed for a specific platform.

You have tools for monitoring your MSSQL databases (such as SQL Monitor) which work well in this platform, but do not play well with others.
To keep things simple, view are in-depth article: Why Simplicity Is Key When Managing Your Complex Database Systems

Lack of Proper Tools

Sometimes you don’t even have tools that do what you need. Similarly you can either purchase tools for the others, or use variants of the above which claim to do this work, but have drawbacks. Maybe each works fine for its specific system.

However, reality being what it is; different sets of DBAs work with different platforms.

Each tool has its own UI, and different processes for reporting data. Maybe one tool can monitor emergencies well, but can’t spot problems before they occur, or if they do, require specialised knowledge to identify potential pitfalls in the logs.

Maybe another system is good at the latter, but doesn’t provide clear enough information on other crucial information, such as whether an index has been dropped, or whether it has just disappeared into the aether never to be seen again.

Missing Overview and Routines

Many of these systems come hermetically sealed. Much like many popular computer companies (who shall be left unnamed) these have beautiful designs and look great out of the box. However, if something doesn’t work that way that you want, there is virtually no way to get under the hood, at least without voiding the complex warranties.

So customisation becomes a hardship.

Others might be a little rougher, but even though you might be able to hack into them, because they use their own proprietary languages, it makes managing these somewhat difficult, and virtually impossible to customise for other systems. 

Home-Grown Tools

Sometimes it’s possible, particularly if you have some particularly talented DBAs, to create some specialised tools. You have a genius working in your systems area who has created their own tools which, through its convoluted way managed to keep records of each system.

She has managed to hack through the proprietary systems (of course voiding the warranty, but hey, she’s talented, who needs warranties anyway, right? Right???) and create something that works.

Unfortunately, she works alone and doesn’t like to share information, and one day gets sick, or rage-quits due to the complexity of all the different systems and the constant changes (you add a MongoDB, and suddenly the idea of a non-
relational NoSQL database system causes her brain to break).

So now you have everyone else left to manage these great tools. Much like the platforms, including the older ones (yes, they keep updating trying to keep up the newer systems, but creating unknown horrors for your DBAs), the tools quickly become no longer compatible.

Due to human limitations (and no, the Singularity has not yet occurred, so you can’t replace your DBAs with robots…. thankfully), keeping all of these scripts and tools up-to-date is a virtual impossibility.

So what do you do?

Ideally what you need is a system that can handle all of these tasks.. If only you could find some tool that could provide a top-level view of all of your systems, with an interface that remains at least mostly consistent, that provides the same information for each of your platforms and versions. If you have the ability to customise the system to handle what you need to do, while at the same time making them accessible and manageable from a central location. Well that would be great, wouldn’t it?